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Abstract 


Expert  support  systems  (ESS)  are  ^systems  designed  to 
help  decision  makers  deal  with  complex,  nonprogrammed 

J'k 

decisions.  This  researeh  entailed  development, 
implementation,  and  validation  of  an  ESS  in  a  "real-world* 
environment.  The  problem  solving  technique  used  here  is  the 
analytic  hierarchy  process,  adapted  to  the  microcomputer  by 


Foreman  and  others  from  Decision  Support  Software  in  the 


form  of  the  software  package  Expert  Choice.  Included  in  the 


research  is  a  background  study  on  decision  making  in  general, 
the  Analytical  Hierarchy  Process,  the  evolution  of  expert 

u.s- 

support  systems,  and  bulled  Slattns  Air  Force  reliability  and 
maintainability  issues. 

The  complex  decision  chosen  for  this  project  was 
Headquarters  Tactical  Air  Command's  annual  prioritization  of 
R&M  modifications.  These ywrtf-  modifications ,  known  as  Class 
IV-B  modifications,  improve  the  effectiveness  of  fielded 
weapon  systems.  Due  to  a  limited  budget,  not  all 
modifications  can  be  funded  in  a  given  year.  Therefore,  TAC 
must  prioritize  the  modifications,  trading  off  benefits 
offered  with  the  cost  of  each  proposed  change.  This  research 
includes  a  model  designed  by  the  author  using  Expert.  Choice 


to  prioritize  these  modifications  that  offer  the  greatest 
benefit/cost  return.  The  model  improves  responsiveness  to 
changes,  increases  flexibility,  and  improves  the  reliability 


of  the  decision-making  process 


•  i  £  (0 


DEVELOPMENT  OF  A  MICROCOMPUTER-BASED 


EXPBRT  SUPPORT  SYSTEM:  APPLICATION  OF  THE 
ANALYTICAL  HIERARCHY  PROCESS  TO  THE 
PRIORITIZATION  OF  RELIABILITY  AND 
MAINTAINABILITY  MODIFICATIONS 

I.  Introduction 

General  Issue 

"Reliability  and  Maintainability  have  long  been 
recognized  as  valid  program  considerations  impacting 
both  combat  capability  and  logistics  supportability 
(Tactical  Air  Command,  1987d:v)." 

Modifications  to  deployed  weapon  systems  are  necessary  to 

improve  their  reliability,  maintainability,  and 

effectiveness,  thus  enhancing  their  combat  capability  and 

logistic  supportability.  Dr.  Benjamin  S.  Blanchard  states  in 

his  book.  Logistics  Engineering  and  Management,  these 

modifications  "...  are  initiated  to  correct  deficiencies 

and/or  to  improve  the  product  (Blanchard,  1986:287).”  For 

the  Tactical  Air  Force  (TAF) ,  these  changes  modify  the 

aircraft  and  weapon  systems  that  are  the  front  line  defense 

of  the  United  States.  To  keep  these  critical  weapon  systems 

fully  effective  and  capable  of  ensuring  our  national  security 

in  light  of  the  constantly  changing  global  threat, 

modifications  are  vital. 


Bach  year  in  the  Program  Objectives  Memorandum  (POM) , 
Headquarters  Air  Force  Logistics  Command  (HQ  AFLC)  submits 
its  proposal  for  funding,  including  a  prioritized  list  of 
modifications  (mods)  to  correct  deficiencies  for  fielded 
weapon  systems.  These  are  known  as  Class  IV  modifications 
and  fall  into  two  categories.  The  first  category,  Class  IV-A 
modifications,  are  related  to  safety  and  will  not  be 
discussed  in  this  research.  The  second,  Class  IV-B 
modifications,  which  increase  the  operational  suitability  of 
aircraft  by  "enhancing  their  reliability  and  maintainability 
(R&M) (Tactical  Air  Command,  1988b: III  5) "  will  be  dealt  with 
here.  Specifically,  the  problems  associated  with  deciding 
which  of  the  Class  IV-B  modifications  to  fund  each  year  from 
a  limited  (and  shrinking)  budget  will  be  the  focus  of  this 
research. 

The  Tactical  Air  Command  (TAC)  Special  Management  Office 
for  Reliability  and  Maintainability  (SMO-R&M)  is  tasked  each 
year  to  prepare  a  prioritized  list  of  all  Class  IV-B 
modifications  for  submission  to  HQ  AFLC  (Tactical  Air 
Command,  1986a:18) .  "SMO-R&M  spearheads  HQ  TAC ' s  effort  to 
enhance  combat  capability  by  increasing  the  reliability  and 
maintainability  of  the  weapon  systems  (Tactical  Air  Command, 
1988b:I-3)."  To  do  this,  each  member  of  the  TAF,  which 
includes  TAC,  the  United  States  Air  Forces  in  Europe  (USAFE) , 
Pacific  Air  Command  (PACAF) ,  Air  National  Guard  (ANG) ,  Air 
Force  Reserve  (AFR) ,  and  the  Alaskan  Air  Command  (AAC) , 
submits  a  list  of  modifications  to  HQ  TAC/SMO-R&M  for 
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consideration.  Each  of  these  commands  has  its  own  interests 
and  priorities  for  the  modifications.  The  task  for  SMO-R&M 
is  to  fold  these  lists  together,  prioritize  the  resulting 
single  list,  and  present  it  to  the  TAF  Commanders  at  their 
fall  commanders  conference.  Following  the  conference,  TAC 
submits  the  commanders  approved  list  to  HQ  AFLC  for  inclusion 
in  the  AFLC  POM  and  Volume  II  of  the  R&M  2000  Plan.  (The  R&M 
2000  Plan  is  revised  annually  "in  order  to  track,  project, 
and  report  R&M  impact  on  the  operational  support  goals  as 
well  as  to  update  the  goals  when  appropriate  (Tactical  Air 
Command,  1987d: 1-1) . " ) 

At  a  time  when  budget  limitations  are  a  reality, 
prioritizing  this  list  is  extremely  difficult.  The  primary 
criterion  for  establishing  the  priority  of  the  modification 
proposals  is  the  benefit  each  offers,  but  overall  cost  and 
Life  Cycle  Costs  (LCC)  must  be  considered  (Collins,  1988b). 
Informally  this  is  referred  to  as  striving  for  the  "most  bang 
for  the  buck,”  and  is  a  complex  problem  requiring  a 
significant  amount  of  time  and  effort  each  year. 

Decision  making  for  managers  and  leaders  of  the  United 
States  Air  Force  (USAF)  is  itself  a  complex  subject.  Many 
factors  are  weighed  in  the  typical  decision  making  process, 
including  needs  of  the  Air  Force,  cost  considerations,  top- 
level  guidance,  adherence  to  applicable  regulations,  and 
common  sense.  Clearly,  prompt,  accurate  decisions  must  be 
made  in  today's  Air  Force  in  spite  of  the  complexities. 
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Expert  Support  System  (ESS)  software,  commercially 


available  at  relatively  low  cost,  is  designed  to  help 
facilitate  managerial  decision  making.  Techniques  to 
structure  problems  and  to  aid  in  decision  making  are  becoming 
more  common  and  are  being  incorporated  into  the  personal 
computer  (PC)  environment.  Air  Force  managers  could  make  use 
of  these  techniques  and  available  low-cost  software  to 
improve  their  decision  making.  The  objective  of  this 
research  is  to  develop  an  ESS  using  such  low-cost  software 
for  use  by  SMO-R&M  in  prioritizing  the  TAF  Class  IV-B 
modifications . 


Problem  Statement 


The  general  problem  addressed  by  this  research  is  that 
managers : 

"are  required  to  make  complex  decisions,  based  on 
competing  criteria,  and  dozens  of  subcriteria,  involving 
numerous  alternative  strategies,  without  the  use  of  a 
comprehensive,  easy-to-use,  decision  support  tool.  ...As 
a  result,  managers  may  make  decisions  without 
considering  all  the  relevant,  important  criteria  that 
influence  the  decision.  In  such  cases,  the  risk  exists 
that  less-than-optimal  decisions  will  be  made... 

(Cook,  1986b: 5).” 

Specifically,  the  Air  Force  can  not  afford  to  make  poor 
funding  decisions.  Changes  or  modifications  to  TAF  aircraft 
for  reliability  and  maintainability  reasons  are  crucial  to 
national  defense.  There  is  a  finite  amount  of  funding 
available  to  accomplish  the  myriad  of  modification  proposals 
requiring  funding.  HQ  TAC/SMO-R&M  must  determine  the 
appropriate  funding  priorities  to  accomplish  these  critical 
modifications,  considering  the  many  cost/benefit  tradeoffs  of 
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the  modifications,  both  individually  and  as  a  group.  This  is 
HQ  TAC’s  method  of  influencing  HQ  AFLC  on  how  modification 
money,  designated  for  TAF  aircraft,  should  be  spent. 

This  research  will  entail  development  of  a  viable 
decision  support  tool  in  the  form  of  an  expert  support  system 
for  use  by  the  SMO-R&M  managers,  and  document  the  application 
of  the  ESS  to  the  specific  problem  stated  above.  The 
following  section  will  describe  TAC's  Fiscal  Year  1991  (FY 
*91)  new  start  Class  IV-B  mod  prioritization  decision 
process.  The  SMO-R&M  managers  are  projecting  these  new 
starts  three  years  in  advance  for  HQ  AFLC's  POM  planning 
purposes.  New  starts,  "...define  new  programs  in  terms  of 
dollars  and  manpower  to  satisfy  a  valid  requirement  (Tactical 
Air  Command,  1987c: II-2) . " 

The  Class  IV-B  Mod  Prioritization  Process 

Following  production  and  deployment  of  a  weapon  system, 
a  mission  profile  change  or  improved  technology  may  make  a 
Class  IV-B  modification  to  the  system  desirable  or  necessary. 
These  modifications  may  be  as  simple  as  removal  of  a  piece  of 
sheetmetal  from  the  belly  of  an  F-15,  or  as  complex  as 
extending  the  life  of  the  Air  National  Guard  and  Reserves’ 
F100-PW-100  and  -200  engines.  SMO-R&M  managers  are  faced 
with  the  complex  reliability  and  maintainability  decisions 
involving  these  critical  weapon  systems. 

Class  IV-B  modifications  can  "nurse”  along  an  aging 
weapon  system  or  create  an  entirely  new  capability  for  these 
systems.  Typically,  a  total  of  $90  to  $100  million  dollars 
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is  annually  allocated  for  these  Class  IV-B  modifications. 
With  all  proposed  FY  '91  new  start  costs  totalling  nearly 
$1.42  billion  dollars,  less  than  ten  percent  of  these 
modifications  will  be  funded  (Collins,  1988a).  The  question 
becomes  one  of  prioritizing  the  "needs"  to  match  available 
funding.  Each  system  program  manager  (SPM)  can  make  a  valid 
case  for  the  weapon  system  he  or  she  is  responsible  for. 

Each  weapon  system  has  a  vital  role  in  the  national  defense. 
Should  many  small  cost  -  small  benefit  mods  be  funded,  or 
should  a  few  more  costly  -  larger  benefit  mods  receive 
priority?  The  objective  is  to  get  the  greatest  benefit  for 
each  dollar  spent. 

The  SMO-R&M  managers  are  currently  using  time  consuming 
manual  methods,  combined  with  costly  mainframe  computer 
resources,  to  prioritize  their  Class  IV-B  mods.  A  basic 
tenet  of  this  research  is  that  SMO-R&M  can  establish  these 
priorities  at  a  significant  saving  of  both  time  and  money 
using  a  PC-based  expert  support  system  developed  by  this 
author.  (All  acronyms  and  abbreviations  used  in  this  work 
can  be  found  in  Appendix  D) 

Research  Objectives 

During  the  course  of  this  research  the  following  steps 
will  be  accomplished: 

1.  Development  of  a  comprehensive  literature  review  of 
military  and  civilian  sources  on: 

A.  Decision  Making. 

B.  Analytical  Hierarchy  Process. 
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C.  Evolution  of  Expert  Support  Systems. 

D.  Reliability  and  Maintainability  issues. 

2.  Research  of  applicable  guidance  and  regulations  for 
the  HQ/TAC  SMO-R&M. 

3.  Development  of  an  expert  support  system  model  to 
help  SMO-R&M  prioritize  the  TAF  Class  IV-B  modifications. 

A.  Demonstration  of  the  model  to  both  the  decision 
makers  and  the  experts. 

B.  Modification  of  the  model  as  necessary. 

C.  Validation  of  the  model  with  data  from  a 
previous  year. 

4.  Documentation  of  the  entire  process. 
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Overview 


This  chapter  will  begin  with  a  general  review  of  all  the 
diverse  subjects  researched.  Then  a  more  thorough  review 
will  follow  of  a  portion  of  the  applicable  literature 
available  on  decision  making,  the  analytic  hierarchy  process 
approach,  the  evolution  of  expert  support  systems,  and  United 
States  Air  Force  reliability  and  maintainability  (R&M) 
issues.  Bach  of  these  topics  will  be  discussed  as  they 
pertain  to  this  research. 

General  Review 

The  subject  nature  of  this  research  has  brought  about 
this  review  of  seemingly  unrelated  subjects.  The  development 
of  a  micro-computer  based  expert  support  system  to  be  used  by 
the  Air  Force  decision  makers  for  R&M  issues  draws  from 
diverse  sources. 

One  of  the  areas  studied  is  decision  making  as  it 
pertains  to  complex  decisions.  The  prioritization  decision 
involves  multiple  criteria,  such  as  benefits,  costs, 
reliability,  and  maintainability  issues,  as  well  as  a 
multitude  of  subcriteria  on  which  the  decision  is  based. 

Another  topic  discussed  is  the  analytic  hierarchy 
process  (AHP)  approach  to  problem  solving.  Developed  by  Dr. 
Thomas  L.  Saaty,  AHP  structures  complex  decisions  for 
decision  makers  (Saaty,  1980b) . 


A  third  area  discussed,  the  evolution  of  expert  support 
systems,  describes  how  computer  software  has  evolved  and  how 
ESS  are  useful  to  decision  makers.  The  more  advanced 
software  systems  can  exercise  computer  power  to  formulate  and 
solve  complex  problems.  The  expert  support  system  used  in 
this  research  -  -  Expert  Choice  -  -  was  designed  by  Dr. 

Ernest  H.  Forman  and  is  based  on  the  analytic  hierarchy 
process  (Foreman  and  others,  1986). 

Finally,  the  Tactical  Air  Force  reliability  and 
maintainability  modification  prioritization  process  is  the 
specific  complex  problem  addressed  by  this  research.  These 
diverse  subjects  tie  together  to  make  for  an  interesting 
research  project  that  has  real  world  applications. 

Decision  Making 

This  research  addresses  two  primary  topics  in  decision 
making.  The  first  area  described  is  how  one  differentiates 
nonprogrammed  and  programmed  decision  aituations.  The  second 
area  is  complex  decisions  and  how  they  are  formulated  to 
reduce  complexity. 

According  to  Nobel  Prize  recipient  Herbert  A.  Simon 
(Fick  and  Sprague,  1980:49)  decisions  are  either  programmed 
or  nonprogrammed  (see  Table  1) .  Programmed  decisions  are 
those  that  are  "...repetitive  and  routine,  to  the  extent  that 
a  definite  procedure  has  been  worked  out  for  handling  them 
(Simon,  1965:58)."  Nonprogrammed  decisions  on  the  other  hand 
are: 

"...novel,  unstructured,  and  consequential.  There  is  no 
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cut-and-dried  method  for  handling  the  problem  because  it 
hasn't  arisen  before,  or  because  its  precise  nature  and 
structure  are  elusive  or  complex,  or  because  it  is  so 
important  that  it  deserves  a  custom-tailored  treatment 
(Simon,  1965:59)." 

This  research  will  focus  on  nonprogrammed  decisions.  The 
fundamental  task  of  prioritizing  a  different  list  of  weapon 
system  modifications  is  clearly  neither  repetitive  nor 
routine. 


Table  1.  Decision  Making 
(Adapted  from  Simon  1965:62) 
TYPES  OF  DECISION 


Programmed:  Routine,  repetitive  decisions. 

Organization  develops. specific 

processes  for  handling  them. 

Traditional  Techniques 

Modern  Techniques 

1.  Habit 

1.  Operational  Research 

2.  Clerical  routine: 

Mathematical  analysis 

Standard  operating 

Models 

procedures 

3.  Organization  structure: 

2.  Electronic  data 

Common  expectations 

processing 

a  system  of  subgoals 

Hell  defined 

informational  channels 

Nonprogrammed:  One-shot,  ill-structured  novel,  policy 

decisions 

Handled  by  general  problem-solving  i 

techniques 

Traditional  Techniques 

Modern  Techniques 

1.  Judgement,  intuition, 

Heuristic  problem-solving 

and  creativity 

technique  applied  to: 

2.  Rules  of  thumb 

(a)  training  human 

3.  Selection  and  training 

decision  makers 

of  executives 

(b)  constructing 

heuristic  computer 

programs 
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"There  is  now  good  reason  to  believe  that  the  process  of 
nonprogrammed  decision  making  will  soon  undergo  as 
fundamental  a  revolution  as  the  one  which  is  currently 
transforming  programmed  decision  making  in  business 
organizations  (Simon,  1965:76)." 

This  "revolution"  is  now  upon  us.  The  personal  computer, 

with  its  enhanced  memory  capabilities,  is  now  the  vehicle  to 

apply  these  "new"  techniques  for  decision  making.  Heuristic, 

according  to  Webster’s  New  Collegiate  Dictionary  is, 

"...providing  aid  or  direction  in  the  solution  of  a 
problem  but  otherwise  unjustified  or  incapable  of 
justification.  Specifically  of  or  relating  to 
exploratory  problem-solving  techniques  that  utilize 
self -educating  techniques  (as  evaluation  of  feedback)  to 
improve  performance  <  a  heuristic  computer  program  > . " 

The  improvement  of  performance  is  the  crux  of  this  decision 

making  problem.  Improved  decision  processes  will  naturally 

improve  the  way  the  AF  does  business. 

Decision  making  for  leaders  and  managers  of  the  United 

States  Air  Force  is  a  complex  subject.  Many  factors  are 

considered  in  the  typical  decision  process,  including  needs 

of  the  Air  Force,  cost  considerations,  top-level  guidance, 

adherence  to  applicable  regulations,  and  common  sense. 

Timely,  rational  decisions  must  be  made  in  today's  Air  Force 

in  spite  of  the  complexities. 

As  Simon  also  points  out,  the  decision  making  process 

for  executives  has  three  phases: 

"The  first  phase  of  the  decision-making  process  - 
searching  the  environment  for  conditions  calling  for 
decision  -  I  shall  call  intelligence  activity  (borrowing 
the  military  meaning  of  intelligence) .  The  second  phase 
-  inventing,  developing,  and  analyzing  possible  courses 
of  action  -  I  shall  call  design  activity.  The  third 
phase  -  selecting  a  particular  course  of  action  from 
those  available  -  I  shall  call  choice  activity 
(Simon,  1965:54)." 
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Each  of  these  activities  or  phases  is  operative  in  this 
research.  The  first  phase,  intelligence  gathering,  is 
performed  by  HQ  TAC/SMO-R&M  "experts"  for  the  TAF.  Two 
separate  HQ  AFLC  data  bases,  discussed  later  in  this  chapter, 
are  needed  to  gather  the  data  required  to  provide  the  TAF 
commanders,  the  decision  makers,  with  the  information 
necessary  to  make  a  decision. 

The  second  phase,  the  design  activity,  is  where  the 
"experts"  design  the  methodology  to  solve  the  problem.  This 
is  done  through  modeling  of  the  environment  in  a  manner 
consistent  with  what  is  actually  found  in  the  "real” 
environment.  Criteria  and  subcriteria  are  selected  that  have 
a  fundamental  effect  on  the  overall  decision.  For  the  1990 
new  start  modification  prioritization,  HQ  TAC/SMO-R&M 
requested  the  Joint  Studies  Group  (JSG)  at  HQ  TAC  to  provide 
a  methodology  to  solve  the  prioritization  problem.  The 
method  the  JSG  proposed  was  Saaty's  AHP .  The  topic  of 
research  by  a  former  AFIT  graduate  student's  (Darko)  thesis 
was  to  validate  AHP  as  a  viable  approach  to  this 
prioritization  process  (Darko,  1987).  This  current  research 
builds  on  this  foundation,  developing  a  responsive  ESS  for 
use  on  a  PC. 

The  third  phase,  the  choice  activity,  selects  the 
particular  course  of  action  from  those  available.  Not  only 
is  the  selection  of  the  possible  alternatives  accomplished  in 
this  phase  but  identification  of  the  "best"  alternative  is 
the  goal.  The  modifications  are  the  alternatives  used  in 
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this  prioritization  decision.  The  modifications  are 
evaluated  with  the  criteria  using  the  priorities  assigned 
when  the  model  was  formulated. 

The  process  of  the  prioritization  of  these  modifications 
is  clearly  not  routine  nor  repetitive  and  cannot  be 
classified,  according  to  the  Simon  definition,  as  a 
programmed  decision.  Therefore  the  techniques  described  in 
Table  1  are  not  suitable  for  this  application.  Nonprogrammed 
decisions  on  the  other  hand  are  those  that  are  handled  by 
general  problem-solving  techniques.  The  heuristic  problem 
solving  technique  applied  in  this  research  is  the  analytic 
hierarchy  process.  AHP ,  applied  in  the  PC  environment, 
allows  responsive  decisions  that  allow  flexibility  to 
facilitate  changes,  while  improving  the  reliability  of  the 
decision  process. 

Often  in  the  Air  Force  environment,  application  of  the 
decision  making  process  is  delegated  to  a  subordinate  who  may 
structure  the  problem,  define  the  alternatives,  and  provide 
the  ultimate  decision  maker  with  the  recommended 
alternatives.  The  decision  maker  then  provides  the 
subordinate  (the  expert)  with  either  approval  of  the 
recommendations  or  "further  guidance.”  "An  Expert," 
according  to  Webster's  New  Collegiate  Dictionary  is,  "one  who 
has  acquired  special  skill  or  knowledge  of  a  particular 
subject."  In  the  Air  Force,  subordinates  at  the  headquarters 
staff  level  positions  are  carefully  chosen  for  their 
expertise . 


Decision  making,  or  any  problem  solving  situation, 
begins  with  a  clear  definition  of  the  problem  or  statement  of 
the  decision  goal,  as  Simon  states.  "Problem  solving  is 
concerned  with  finding  paths  from  initial  states  to  desired 
states  (Newell  and  Simon,  1972:828)."  According  to  Dr. 

Curtis  R.  Cook,  "To  optimize  the  decision-making  process, 
all  relevant  criteria  and  alternatives  must  be  identified 
(Cook,  1987a:31).”  These  criteria  and  subcriteria,  along 
with  the  alternatives,  can  be  easily  structured  in  a  decision 
hierarchy. 

Analytic  Hierarchy  Process 

As  stated  above,  this  process  of  structuring  complex 
problems  was  designed  by  Dr.  Thomas  L.  Saaty  and  termed  the 
Analytic  Hierarchy  Process  (AHP) .  Saaty  believed  decision 
makers  needed  a  process  to  reduce  "formidably  intricate 
systems  to  a  sequence  of  pairwise  comparisons  of  properly 
identified  components  (Saaty,  1980b:4)."  Once  defined  in  the 
hierarchy,  a  series  of  pairwise  comparisons  can  be  made 
between  alternatives  by  decision  makers  at  the  lowest  level 
of  the  hierarchy. 

A  hierarchy  is  defined  as  a  "top-down  structure  of  a 
complex  problem  that  makes  it  easier  to  provide  judgement  and 
make  choices  (Foreman  and  others,  1986:G-5)."  At  the  top  of 
the  process  a  goal  to  be  attained  is  defined.  Then  all 
criteria  that  influence  the  decision  are  defined  along  with 
any  relevant  subcriteria.  In  this  manner,  the  problem  is 
structured,  and  the  alternative  courses  of  action  are  placed 
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at  the  lowest  level  of  the  decision  hierarchy.  Figure  1 
presents  a  typical  hierarchy.  The  criteria  and  subcriteria 
are  determined  by  the  decision  maker.  Each  of  the  criteria 
is  compared  to  "peer"  criteria  during  pairwise  comparisons  to 
establish  a  rank  ordering  or  priority  of  the  criteria. 
Subcriteria  are  handled  in  the  same  way  to  give  each  node 
"weights”  that  sum  to  1.0  for  the  entire  hierarchy.  At  the 
lowest  level  of  the  hierarchy,  each  of  the  N  (some  number  of) 
alternatives  is  compared,  "one  at  a  time  against  each  other, 
with  respect  to  their  relative  impact  (Saaty,  1982a:240),"  on 
each  of  the  criteria  and  the  subcriteria. 

This  process  approach  to  complex  problem  solving  was 
chosen  for  this  research  because  in  1987,  HQ  TAC/SMO-R&M 
sponsored  research  from  an  AFIT  operational  research  student, 
First  Lieutenant  Katherine  Darko.  The  objective  of  her 
thesis  was  to  validate  AHP  as  a  viable  process  for  HQ  TAC  to 
use  to  prioritize  Class  IV-B  modifications  (Darko,  1987) . 
Though  not  completely  satisfied  that  AHP  was  the  optimal 
approach  for  TAC  to  use  in  this  application,  her  research  was 
insightful . 

Darko  pointed  out  in  her  research  that  the  TAF 
commanders  did  not  have  the  opportunity  to  structure  the 
decision  tree.  This  is  a  time-consuming  process. 

Structuring  the  problem,  and  performing  all  pairwise 
comparisons  of  criteria,  subcriteria,  and  alternatives,  is  a 
long  and  tedious  process  when  performed  manually.  The 
commanders'  time  is  limited,  and  there  are  other 
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Figure  1.  The  Decision  Hierarchy 


prioritization  lists  to  be  considered.  As  a  result,  the 
structuring  of  the  problem  and  the  weighting  of  the  nodes  is 
accomplished  by  the  special  management  office  for  R&M.  These 
"experts"  have  the  necessary  skill  and  expertise  to  provide 
and  coordinate  the  various  portions  of  the  tree  with  the 
decision  makers  and  their  staffs. 

Prior  to  the  application  of  AHP  to  this  prioritization 
process,  the  manual  process  was  too  complex,  and  may  have 
resulted  in  satisficing.  Newell  and  Simon  define  satisficing 
as,  "setting  an  acceptable  level  or  aspiration  level  as  final 
criterion  and  simply  taking  the  first  acceptable  move 
(alternative) (Newell  and  Simon,  1972:681).”  If  satisficing 
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did  result  from  the  overwhelming  complexity,  poor  funding 
priorities  may  have  resulted  to  the  extent  that  national 
security  may  have  been  impacted.  Darko's  treatment  of  AHP  is 
a  start,  but  the  process  is  still  much  too  cumbersome 
manually.  It  takes  too  much  time  to  coordinate  with  the 
Joint  Studies  Group  analysts,  not  to  mention  the  amount  of 
time  required  for  mainframe  computer  support.  TAF  commanders 
need  better  response  to  make  good  prioritization  decisions. 
This  research,  application  of  AHP  on  a  microcomputer,  affords 
the  experts  the  flexibility  to  give  fast,  rational 
recommendations  to  the  decision  makers.  The  current  effort 
differs  from  Darko’s  because  it  will  provide  a  viable  PC- 
based  software  tool  for  the  SMO-R&M  managers  to  use  in  making 
decisions . 

Air  Force  "microcomputer  acquisitions  have  increased 
tremendously  since  1983  (Van  Scotter,  1986:22),"  states 
Captain  James  Van  Scotter.  With  the  availability  of  these 
microcomputers  to  most  AF  managers,  computer  software  tools 
to  effectively  use  these  powerful  tools  in  decision  making 
are  needed. 

Evolution  of  Expert  Support  Systems 

Computer  software  systems  have  evolved  from  simple 
office  automation  systems,  electronic  data  processing, 
management  information  systems,  to  more  complex  decision 
support  systems,  expert  systems,  and  expert  support  systems 
(see  Table  2).  Each  of  these  software  systems  was  designed, 
to  accomplish  certain  tasks  for  specific  management  levels. 


Table  2. 

Software  Systems 

User 

Software  Systems 

Top  Management 

Expert  Systems,  Expert 

Support  Systems,  and  Decision 
Support  Systems 

Middle  Management 

Management  Information  Systems 

First  Line  Management 

Electronic  Data  Processing 

Clerical  Personnel 

Office  Automation  Systems 

Once  designed  these  software  systems  were  made  available  to 
all  users  at  any  level,  who  could  justify  their  particular 
requirement  for  computer  support. 

Clerical  Personnel  Needs .  Office  automation  systems  are 
those  computer  systems  that  facilitate  office  work,  such  as 
word  processing.  These  systems  were  designed  to  automate 
routine  office  work  for  clerical  personnel. 

First  Line  Management  Needs.  Electronic  data  processing 
(EOP)  was  designed  to  process  fundamental  repetitive  tasks 
found  at  this  level  in  the  organization.  According  to 
Sprague  and  Carlson,  "EDP  was  first  applied  to  lower 
operational  levels  of  the  organization  to  automate  the 
paperwork  (Sprague  and  Carlson,  1982:6)." 

Middle  Management  Needs.  Middle  management  needed  the 
data  integration  of  management  information  systems  (MIS) . 

MIS  draw  from  various  data  bases  to  provide  information 
needed  by  this  level  in  the  organization.  These  systems  were 
designed  with  integration  and  information  management  in  mind. 
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Top  Management  Needs.  Many  computer  software  systems 
are  designed  to  facilitate  this  level  in  the  organization. 
Beginning  with  decision  support  systems,  and  ending  with 
expert  support  systems  this  research  will  briefly  discuss 
each  of  these  areas  as  they  pertain  to  top  management. 

Decision  Support  Systems  (DSS)  are  computer  systems 
aimed  at,  "..quickly  assisting  managers  in  making  effective 
decisions  in  those  areas  where  both  management,  judgement, 
and  computer  analysis  are  required  (Allen  and  Emmelhainz, 
1984:129)."  According  to  Dr.  Richard  E.  Pesche  in  the  book 
Military  Logistics, 

"The  emphasis  of  a  DSS  is  on  "customized  support”  for 
the  decision  maker.  A  DSS  addresses  four  major  concerns 
of  the  decision  maker.  These  include: 

1.  SEMI -STRUCTURED  TASKS  -  A  DSS  addresses  semi- 
structured  tasks  which  include  explicit,  well 
defined  and  algorithmic  procedures  as  well  as 
intuitive  and  subjective  procedures. 

2.  SUPPORT  -  A  DSS  is  intended  to  help  the  manager 
do  his/her  job. 

3.  EFFECTIVENESS  -  A  DSS  must  be  capable  of 
identifying  what  should  be  done  and  assuring 
that  the  selected  problem  solving  approach  and 
criteria  are  relevant  to  the  issue  at  hand. 

4.  EFFICIENCY  -  The  DSS  must  be  capable  of 
performing  a  task  as  efficiently  as  possible  in 
relation  to  a  pre-established  procedure 
(Westfall  and  others,  1987:502)." 

An  expert  systems  is  defined  as: 

"...a  computer  program  that  mimics  a  human  expert;  using 
the  methods  and  information  acquired  and  developed  by  a 
human  expert,  an  expert  system  can  solve  problems,  make 
predictions,  suggest  possible  treatments,  and  offer 
advice  with  a  degree  of  accuracy  equal  to  that  of  its 
human  counterpart  (Allen  and  Emmelhainz,  1984:136)." 
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Expert  systems  coupled  with  elements  of  decision  support 
systems  form  the  basis  for  Expert  Support  Systems  (ESS) . 
Expert  system  knowledge  bases  are  reservoirs  of  information 
consisting  of  logical  relationships  and  deductive  rules  that 
affect  the  likelihood,  preference,  and  characteristics  of 
various  outcomes  under  different  conditions  (Davis,  1988:64). 
Dr.  Ernest  H.  Foreman  makes  a  differentiation  between  ESS  and 
ES.  He  states, 

"An  Expert  Support  System  differs  from  an  Expert  System 
in  that  the  former  is  typically  designed  to  support  an 
expert  where  the  latter  is  designed  to  replace  an  expert 
(Foreman,  1986:457)." 

These  are  not  subtle  differences  (see  table  3) .  Expert 
support  systems  augment  an  expert  during  the  chore  for  which 
it  was  designed  —  in  this  case,  our  prioritization  problem. 
Foreman  further  states: 


Table  3. 

Expert  Systems  vs 
(Adapted 

Expert  Support  Systems 
from  Foreman,  1986:457) 

Expert  Systems 

Expert  Support  Systems 

Frequency  of 
Problem 

Frequently 

One  of  a  kind 

Level  of 
Expertise 
required 

Up  to  Expert 
e.g.  diagnosis, 
filling  compli¬ 
cated  forms 

Beyond  Expert 
e.g.  strategic, 
policy 

Benefit 

Reduced  costs 
Increased  Speed 

Better  decisions 
Improve  communication 

20 


"Until  recently,  multi  criteria  problems  were  extremely 
difficult  to  deal  with.  However,  with  the  advent  of  the 
Analytical  Hierarchy  Process  (AHP)  and  the  personal 
computer,  multi-criteria  problems  can  be  addressed  in  a 
rational  manner  (Foreman,  1986:456)." 

Expert  Choice  is  an  ESS  designed  by  Dr.  Foreman  to  augment 

experts  in  structuring  and  solving  their  problems.  This 

software  helps  the  decision  maker  perform  the  pairwise 

comparisons  inherent  in  Saaty's  AHP,  and  vital  to  the 

prioritization  problem.  The  pairwise  comparisons  will  be 

demonstrated  in  the  next  chapter.  Expert  Choice  also  has  the 

ability  to  tie  spreadsheets,  text,  and  raw  data  to  any 

specific  node  for  documentation  purposes. 

Artificial  Intelligence  has  not  been  discussed  in  this 

research  because  in  this  context,  the  ESS  developed  for  this 

prioritization  process  is  not  considered  AI. 


Reliability  and  Maintainability  Issues 

Air  Force  reliability  and  maintainability  issues  are 
complex: 

"As  we  accumulate  field  experience  with  new  weapon 
systems,  it  may  become  apparent  that  a  system's  R&H 
characteristics  fail  to  meet  expectations  through 
mis judgment,  mismanagement,  changes  in  operating 
environment,  or  other  unforeseen  causes.  This 
experience  often  leads  to  system  modification  proposals 
and  projects  so  as  to  correct  or  enhance  R&M 
characteristics  (Westfall  and  others,  1987:400)." 

Dr.  Benjamin  S.  Blanchard  defines  reliability  as: 

"...  the  probability  that  a  system  or  product  will 
perform  in  a  satisfactory  manner  for  a  given  period  of 
time  when  used  under  specified  conditions  (Blanchard, 
1986:12) ." 


21 


Department  of  Defense  (DOD)  Directive  5000.40  defines 
maintainability  as: 

"The  ability  of  an  item  to  be  retained  in  or  restored  to 
a  specified  condition  when  maintenance  is  performed 
having  specified  skill  levels,  using  prescribed 
procedures  and  resources . " 

Class  IV-B  modifications  are  those  mods  that  "modify  to 
correct  safety  or  R&M  deficiencies  (Tactical  Air  Command, 
1987c:II-9) . "  Safety  modifications  (Class  IV-A)  are 
automatically  given  top  priority,  and  therefore  funded  first. 
Class  IV-B  mods  to  implement  R&M  are  the  focus  of  this 
research. 

HQ  TAC/SMO-R&M  is  annually  tasked  to  prioritize  the  TAF 
Class  IV-B  modifications  (Tactical  Air  Command,  1986a: 18; 
Tactical  Air  Command,  1988b: III-5;  Tactical  Air  Command, 
1987d:I-15) .  Data  from  two  of  HQ  AFLC's  data  bases  are 
required  to  establish  funding  priorities.  The  first  data 
base,  the  Pacer  Lab  System,  is  required  to  provide  cost 
information.  The  second  data  base  is  the  Maintenance  and 
Operational  Data  Access  System  (MODAS) .  This  system  provides 
information  on  the  aircraft  affected  by  the  proposed 
modification  (Yetter,  1988) .  Once  all  information  from  these 
data  bases  is  retrieved  by  SMO-R&M  personnel,  it  is  arranged 
on  modification  worksheets  (refer  to  Figure  3  chapter  4) ,  and 
input  into  the  AHP  model  operated  by  HQ  TAC/JSG. 

The  Joint  Studies  Group  analysts,  in  conjunction  with 
the  SMO-R&M  experts,  build  the  TAF  decision  hierarchy  model 
as  demonstrated  in  figure  2.  This  AHP-based  model  is 
designed  to  run  on  JSG's  mainframe  computer.  Communication 
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1988  R&M  Mod  Decision  Hierarchy 


Reliability 

Maintainability 

G  0.25 

G  0.25 

Imp  Rank 

G  .125  G  .125 


Imp  -Rank 
G  .125  G  .125 
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G  -  Global  Priority  Relative  to  the  Goal 
Figure  2.  1988  R&M  Mod  Decision  Hierarchy 
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of  the  complexities  of  the  decision  processes  between  SMO-R&M 
and  the  computer  analysts  (JSG)  is  crucial  to  providing  the 
decision  makers  with  the  best  possible  results.  Once  all  the 
modification  worksheet  data  is  entered  into  the  model,  the 
model  is  executed.  The  resulting  list  of  prioritized 
modifications  is  provided  to  SMO-R&M. 

The  final  prioritized  list  is  provided  to  the  TAF 
commanders  at  the  fall  TAF  commanders  conference  for 
approval.  Once  approved  the  list  is  sent  to  HQ  AFLC  to  be 
considered  when  HQ  AFLC  combines  the  lists  from  the  other 
major  commands  for  HQ  AFLC's  POM  submission.  The  list  is 
also  published  in  Volume  II  of  the  R&M  2000  Plan. 

Headquarters  AFLC  uses  a  prioritization  model  known  as 
HO40  to  prioritize  all  Air  Logistic  Centers  (ALC)  proposals 
for  modifications  on  all  weapon  systems  in  the  AF  inventory. 
This  model  was  developed  in  1979,  to  be  used  for  budget 
purposes  (Kalfas,  1988).  The  major  commands  along  with  the 
SPMs  were  not  satisfied  with  the  priorities  of  modifications 
based  on  the  HO40's  goal,  i.e.  funding  executability .  This 
means  the  lower  cost  modifications  are  given  priority  over 
more  costly  modification  proposals.  The  model  algorithm  is 
also  heavily  weighted  toward  "Doability" ,  that  is,  the 
probability  that  funds  allocated  will  be  obligated.  The 
doability  is  often  emphasized  at  the  expense  of  user 
"desirability"  and  the  easiest/cheapest  modifications  are 
done  ahead  of  higher-priority,  more  critical  mods  (Tactical 
Air  Command,  1986a: 19).  The  MAJCOMs  (the  "users")  and  the 
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SPMs  (the  weapon  system  managers)  argue  that  the  performance 
measured  goals  should  have  the  priority  over  funding 
executability  considerations.  HQ  TAC's  method  of  influencing 
this  process  is  to  collect  all  proposed  modifications, 
prioritize  the  list,  publish  the  list  in  the  R&M  2000  Plan, 
and  send  the  list  to  HQ  AFLC . 

HQ  AFLC  is  working  toward  using  the  weapon  systems 
master  plan  for  the  modification  priorities  proposals 
(Kalfas,  1988).  Section  1  of  the  Weapon  Systems  Master  Plan 
document  contains  the  prioritized  list  of  all  modifications 
for  the  weapon  system.  The  weapon  system  master  plan  is 
coordinated  between  the  SPMs  and  the  MAJCOMs  (Witherell, 

1988) .  Once  coordinated  this  document  presents  a  "united 
voice”  to  Headquarters  Air  Force  providing  the  direction  for 
the  weapon  system  management  (Kalfas,  1988) .  HQ  AFLC  is 
still  concerned  with  the  ability  of  the  proposed 
modifications  to  be  able  to  begin  once  funding  becomes 
available.  This  proposed  change,  the  usage  of  the  weapon 
system  master  plan,  will  occur  once  all  MAJCOMs  and  SPMs 
agree  on  the  prioritization  procedure  in  the  coordination 
process.  It  is  hoped  that  the  author’s  AHP  based 
modification  prioritization  expert  support  system  will  be 
adopted  or  will  play  a  part  in  the  proposed  change. 

The  Air  Force  Reliability  and  Maintainability  Policy, 

AFR  800-18  describes  the  operational  R&M  goals.  Listed  in 
order  of  priority  the  goals  are: 

a.  Increase  Combat  Capability.  Increase  operational 
capability,  sustainability,  suitability,  and 


< 


t 


i 


a 

¥ 

i 

■ 

■ 


25 


probability  of  mission  success  by  acquiring  systems 
that  break  infrequently  and  are  easily  and  quickly 
repaired. 

b.  Increase  the  survivability  of  the  combat  support 
structure.  Reduce  or  eliminate  elements  of 
maintenance  and  support  structure  subject  to  attack 
and  destruction,  and  improve  the  ability  of  the  unit 
to  disperse  for  survivable  operation. 

c.  Decrease  mobility  requirements  per  unit.  Reduce  or 
eliminate  airlift  requirements  for  deploying  units, 
and  support  requirements  for  ground  mobile  units. 

d.  Decrease  manpower  requirements  per  unit  of  output. 
Ensure  that  systems  can  be  operated  and  maintained 
with  minimum  personnel,  specialties,  and  skill 
levels . 

e.  Decrease  costs.  Decrease  R&M-driven  costs 
(Department  of  the  Air  Force,  1986a: 1). 

Along  with  the  R&M  2000  goals  R&M  priorities  are  outlined  in 

AFR  800-8. 

Air  Force  Regulation  800-8,  Integrated  Logistics  Support 
(ILS)  Program  states  that  "Reliability,  maintainability,  and 
supportability  requirements  and  related  resources  must  be 
considered  with  cost,  schedule,  and  performance  requirements 
(Department  of  the  Air  Force,  1986b:l). 

When  designing  the  decision  hierarchy  the  model  builders 
must  consider  both  of  these  regulations.  Criteria  and 
subcriteria  must  be  weighted  to  give  the  appropriate  measure 
of  conformance  to  these  directives. 


Summary 

This  chapter  presented  many  diverse  subjects  with  a 
common  thread  of  purpose.  That  purpose  was  the  basis  of  this 
research  -  to  develop  a  microcomputer-based  expert  support 
system,  while  applying  the  analytic  hierarchy  process  to 
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prioritize  HQ  TAC/SMO-R&M* s  Class  IV-B  modifications.  Though 
not  a  totally  encompassing  literature  review  on  any  one  of 
the  relevant  subjects,  the  subjects  were  treated  as  they 
pertain  to  this  research. 

The  next  chapter  will  describe  the  methodology  used  in 
this  research  to  study  the  complex  decision  making  process  HQ 
TAC  currently  accomplishes. 
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Overview 


III.  Methodology 


This  chapter  will  describe  a  complex  decision  process  and 
an  expert  support  system  developed  to  structure  and  solve  the 
research  problem.  The  research  specifically  addresses  the 
complex  decision  of  prioritization  that  HQ  TAC  faces. 

Annually  HQ  TAC/SMO-RSM  prioritizes  the  Class  IV-B 
modifications  for  the  TAF.  This  prioritized  list  then  is 
sent  to  HQ  ALFC  to  be  combined  with  other  major  air  commands 
lists  for  the  POM  submission.  This  is  HQ  TAC's  method  of 
influencing  HQ  AFLC  on  how  modification  money  designated  for 
TAF  aircraft  should  be  spent.  This  chapter  will  describe  the 
general  research  methodology  and  HQ  TAC’s  process  to  gather 
the  information. 

Research  Design 

The  research  design  used  for  this  project  is  Action 

Research.  The  purpose  of  this  design  is: 

"To  develop  new  skills  or  new  approaches  and  to  solve 
problems  with  direct  application  to  the  classroom  or 
working  world  setting  (Isaac  and  Michael,  1978:27).” 

Action  Research  has  many  advantages  over  other  research 

designs.  This  research  will  have  direct  application  to  the 

working  world  and  will  also  provide  an  "...orderly  framework 

for  the  actual  observations  used  to  develop  and  solve  the 

problem  (Isaac  and  Michael,  1978:45).”  Allowing  changes  to 

be  made  during  the  process,  this  design  is  "...flexible  and 


adaptive  to  the  changing  needs  of  the  user  (Isaac  and 
Michael,  1978:45)",  and  does  not  relegate  the  research  to  a 
rigid  structure. 

The  Action  Research  design  is  limited  in  generalized 
application,  as  well  as  external  and  internal  validity  due  to 
the  small  sample  size  used  in  this  research.  With  only  one 
organization  responsible  for  this  prioritization  process,  the 
effects  of  the  ESS  may  not  be  generalized  to  other 
organizations  which  may  be  using  the  same  or  similar 
processes.  The  one-group  pretest-posttest  design  discussed 
by  Campbell  and  Stanley  was  not  chosen  for  this  research 
because  the  lack  of  data  points  for  the  posttest  would  not 
allow  statistical  inference  to  be  made  to  the  general 
population  (Campbell  and  Stanley,  1960:45). 

HQ  TAC  *  s  Process 

The  following  is  a  discussion  of  Headquarters  TAC ' s 
decision  making  process  in  the  arena  of  Class  IV-B  mod 
prioritization,  observed  by  the  author  on  a  trip  to  Langley 
AFB ,  Virginia  in  June  1988. 

SMO-R&M's  goal  in  producing  a  TAF  commanders  approved 
list  of  modifications  is  to  influence  and  defend  mods  as  they 
are  listed  on  the  AFLC  (H040)  composite  list  in  their  POM 
submission  and  in  the  HQ  TAC  R&M  2000  Plan  Volume  II 
(Collins,  1988a).  The  objectives  are  to: 

1.  Offset  the  bias  of  executability  imposed  oy  the  HO40 
Model  and  keep  the  users’  need  in  focus. 

2.  Be  a  readily  available  tool  for  Headquarters  Air 
Force  to  consult  when  going  through  budget  change 
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exercises.  Allows  the  office  to  better  defend  or 
have  an  opportunity  to  defend  the  high  priority  mods 
(Collins,  1988b : 1 ) . 

Annually  HQ  TAC  is  the  focal  point  for  acquiring  the 
information  necessary  to  prioritize  the  TAF  Class  IV-B 
modifications.  The  Special  Management  Office  for  Reliability 
and  Maintainability  (SMO-R&M)  at  HQ  TAC  has  the 
responsibility  to  gather,  coordinate,  prioritize,  and  submit 
the  TAF  commanders  approved  list  to  HQ  AFLC.  This  list  is 
then  used  in  the  HQ  AFLC  POM  submission  and  Volume  II  of  the 
R&M  2000  Plan. 

The  process  begins  when  HQ  TAC/SMO-R&M  requests  the 
field  to  submit  all  of  their  modification  requirements.  This 
is  done  by  a  message  requesting  the  TAF  organizations  submit 
all  known  modifications. 

The  special  project  manager  (SPM)  for  a  particular 
weapon  systems  may  elect  to  prioritize  his  or  her  own  list  of 
modifications  to  influence  the  SMO-R&M  prioritization.  The 
F-16  and  the  F-15  SPMs  have  stated  they  will  prioritize  their 
modification  lists  before  submission  to  HQ  TAC.  Once 
submitted,  the  mod  list  to  SMO-R&M  will  be  rank  ordered  in 
the  model  as  the  SPM  requested. 

Once  all  mod  requests  are  received  by  message  or  through 
the  SPM's,  mod  worksheets  (see  figure  3)  are  filled  out  for 
each  mod  by  SMO-R&M.  Each  of  the  data  elements  on  the  mod 
worksheet  is  described  below: 

1.  Aircraft  Impact:  Based  on  expected  performance  of 
the  mod  to  avert  a  possible  aircraft  grounding 
situation. 
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User/SPM  Ranking 

SHORT  TITLE:  _ 

MOD  NUMBER :  _ 

DESCRIPTION: 


1.  Does  this  mod  avert  a  possible  aircraft 
grounding  situation  {High,  Medium,  Low  or 

No  potential) :  _ 

2.  Reliability  Improvement:  G063  MTBM  Predicted/Current 

Predicted:  _ 

Current:  _  Ratio:  _ 


3.  Reliability  Ranking  Factor: 

(G063  MTBM  Current  Data) 

4.  Maintainability  Improvement: 

(GO 6 3  Current  Data) 

5.  Maintainability  Ranking  Factor: 
(G063  MMH/FH  Current  Data) 

6.  Aircraft  Year  Group: 

(Mean  production  Year  for  Group  of 
Affected  Aircraft) 

7.  Inventory  Size: 

(Number  of  Aircraft  Affected) 

8.  Mod  Cost: 

(AFLC  Pacer  Lab  P3B  Data  -  Total) 

9.  Payback  Period: 

(Is  there  an  amortization  schedule) 

OTHER  REMARKS :  _ 


Figure  3.  Mod  Worksheet 


2.  Reliability  Improvement:  Ratio  of  predicted  mean 
time  between  maintenance  (MTBM)  versus  current  G062 
MTBM  data  found  on  the  MODAS  data  base.  The  greater 
the  predicted  improvement,  the  greater  the  benefit 
realized. 

3.  Reliability  Ranking  Factor:  Measure  of  how  poor  the 
reliability  of  a  system/component  is  as  compared  to 
all  systems  that  make  up  the  weapon  system,  i.e.  a 
measure  of  "hurt”.  Therefore,  the  higher  the 
ranking  factor  (percentile) ,  the  greater  the  need  to 
demonstrate  improvement  for  the  existing  system. 

4.  Maintainability  Improvement:  Predicted 
maintainability  improvement  for  the  system,  the 
greater  the  improvement  the  greater  the  benefit 
realized.  Reflects  increased  aircraft  time  and  fix 
rate. 

5.  Maintenance  Ranking  Factor:  Measure  of  how  poor  the 
maintainability  of  a  system/component  is  as  compared 
to  all  systems  that  make  up  the  weapon  system,  i.e. 

a  measure  of  the  high  man-hour  drivers  that  are 
"eating  your  lunch."  Therefore,  the  higher  the 
ranking  factor  the  greater  the  need  to  demonstrate 
improvement  for  the  existing  system. 

6.  Aircraft  Year  Group:  Based  on  the  mean  production 
year  for  the  group  of  effected  aircraft.  Savings 
accrue  longer  for  improvements  and  impact  of 
improvements  are  more  lasting. 

7.  Inventory  Size:  Based  on  the  number  of  units  in 
service.  Cost  of  mod  is  lowered  by  distributing 
developmental  costs  of  over  a  larger  base.  Cost 
savings  through  improvement  proved  R&M  increases 
with  size  of  inventory  involved. 

8.  Mod  Cost:  Based  on  AFLC  Pacer  Lab  data  base  P3B 
data,  reflects  the  total  cost  of  the  mod  including 
spares  and  support  equipment. 

9.  Payback  Period:  Time  required  to  offset  money 
expended  to  benefit  from  the  mod  is  realized.  '  The 
sooner  the  payback  the  better  (Collins,  1988a). 

These  data  elements  are  directly  related  to  the  R&M  2000 

goals  explained  in  the  last  chapter.  The  data  elements  and 

how  they  relate  are  as  follows: 

1.  Aircraft  Impact:  Should  an  aircraft  fleet  or  weapon 
system  not  receive  the  benefits  of  the  modification 


and  become  unusable  (grounded) ,  combat  capability  of 
the  weapon  system  is  naturally  negatively  affected. 

2.  Reliability  Improvement:  This  directly  impacts  the 
combat  capability  of  the  aircraft  or  weapon  system 
the  modification  is  intended  for. 

3.  Reliability  Ranking  Factor:  The  ability  of  a  weapon 
system  to  perform  as  required  is  a  factor  of  mission 
capability  and  combat  capability. 

4.  Maintainability  Improvement:  Reduction  of  the 
amount  of  time  required  to  fix  an  aircraft  places 
the  aircraft  back  to  a  mission  capable  status 
quicker  impacting  combat  capability. 

5.  Maintenance  Ranking  Factor:  Reducing  the  highest 
probable  failure  item  on  an  aircraft  will  improve 
the  combat  capability  of  the  entire  aircraft  fleet. 

6.  Aircraft  Year  Group:  A  function  of  improving  the 
weapon  system  that  will  receive  the  greatest  overall 
benefits  from  the  modification  will  impact  not  only 
the  cost  considerations,  but  may  decrease  the  number 
of  men  required  to  service  the  aircraft  for  the  LCC. 

7.  Inventory  Size:  Improving  the  largest  number  of 
aircraft  will  demonstrate  the  greatest  cost  savings 
and  greatest  manpower  reduction. 

8.  Modification  cost:  While  the  greatest  benefit  is 
the  desired  goal  the  total  mod  cost  is  considered  in 
the  model  but  not  a  function  of  the  R&M  2000  goals. 

9.  Payback  Period:  The  amortized  period  where  benefits 
from  the  mods  are  realized,  following  the  costs 
expended . 

While  not  all  mod  proposals  will  have  the  maximum  R&M  2000 
impact,  all  proposals  that  can  be  demonstrated  to  have 
savings  in  any  of  these  areas  are  rewarded  in  the  model. 

Information  is  then  gleaned  from  two  of  AFLC's 
information  data  bases  and  added  to  the  worksheets.  The 
first  data  base  is  the  P3B  exhibit  from  the  Pacer  Lab  Data 
Base  System.  The  second  is  the  Maintenance  and  Operational 
Data  Access  System  (MODAS)  data  base,  referred  to  as  G063 
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data.  The  mod  worksheets  with  the  data  elements  required  to 
make  the  decision  are  then  submitted  to  the  Joint  Studies 
Group  ( JSG)  at  HQ  TAC  for  their  computer  analysis. 

The  structuring  of  the  problem  in  the  form  an  analytic 
hierarchy  (referred  to  after  this  as  the  1988  R&M  Mod 
Decision  Hierarchy  (see  figure  4)),  is  the  task  of  the 
experts  in  SMO-R&M  along  with  the  analysts  in  JSG.  They 
prioritize  the  criteria  and  the  subcriteria  for  each  node  of 
the  hierarchy  that  the  decision  makers  will  require  using  the 
pairwise  comparison  technique  (Schooff,  1983).  Benefit  is 
compared  to  cost;  aircraft  impact  is  compared  to  aircraft 
year  group,  inventory  size,  and  R&M  benefit;  modification 
cost  is  compared  to  LCC;  reliability  improvement  is  compared 
to  reliability  rank;  and  maintainability  improvement  is 
compared  to  maintainability  rank.  Based  on  the  judgement  of 
the  experts  mentioned  above,  the  1988  model  was  designed  with 
benefits  four  times  more  important  than  cost  (0.8  to  0.2) 
(refer  again  to  figure  4) .  Under  the  benefit  portion  of  the 
hierarchy,  aircraft  impact  (A/C  Imp)  was  factored  three  times 
that  of  aircraft  year  group  (A/C  Yr  Gp) (0.15  to  0.05); 
Inventory  size  (Inv  Size)  was  given  twice  the  importance  of 
A/C  Yr  Gp  (0.1  to  0.05);  and  R&M  benefit  was  ten  times  that 
of  A/C  Yr  Gp  (0.5  to  0.05).  Under  the  R&M  benefit, 
reliability  and  maintainability  were  ranked  equally  with  an 
overall  value  of  0.25.  Improvement  and  rank  under  both 
reliability  and  maintainability  were  equally  factored  and 
given  specific  weights  of  0.125  each.  Under  the  cost  portion 
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1988  R&M  Mod  Decision  Hierarchy 


Reliability 
G  0.25 


Imp  Rank 

G  .125  G  .125 


Maintainability 
G  0.25 


' Imp  Rank  i 
G  .125  G  .1251 


-  1 

-  1 

-  1 

—  to 

—  to 

—  to 

—  N 

—  N 

—  N 

G  -  Global  Priority  Relative  to  the  Goal 
Figure  4.  1988  R&M  Mod  Decision  Hierarchy 
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of  the  hierarchy,  modification  cost  (Mod  Cost) ,  and  payback 
period  were  given  equal  weights  of  0.1.  A  decision  hierarchy 
using  Dr.  Thomas  L.  Saaty's  analytic  hierarchy  process  (AHP) 
is  used  by  the  JSG  to  order  the  problem  and  derive  a  solution 
(Collins ,  1988a) . 

Each  of  the  TAP  organizations  is  then  sent  the 
prioritized  list  to  coordinate  with  the  respective  commands. 
The  feedback  gained  from  these  commands  on  their  mods  ensures 
that  no  mods  are  inadvertently  left  off  the  final  list  or 
that  priorities  are  not  grossly  in  error. 

Briefings  are  then  prepared  by  the  SMO-R&M  personnel  for 
the  fall  TAP  commanders  conference.  Since  each  command  has 
already  coordinated  the  prioritization  process  through  the 
respective  chain-of -command,  approval  of  the  list  is 
simplified. 

HQ  TAC/ SMO-R&M  submits  the  list  to  HQ  AFLC  and  publishes 
Volume  II  of  the  R&M  2000  Plan. 

HQ  TAC*s  Improvements 

At  the  1987  TAP  commanders  conference  the  commanders 
made  pairwise  comparisons  for  each  of  the  decisions  for  all 
the  proposed  Class  IV-B  mods,  using  the  analytic  hierarchy 
process,  based  on  manual  and  mainframe  methods.  Mod  1  was 
compared  to  mod  2,  3,  4,  5,  ...  N  (some  number  of 
modifications),  then  mod  2  was  compared  with  mod  3,  4,  5,  ... 
N,  etc.,  for  the  entire  list  of  modifications.  This  process 
proved  to  be  very  time  consuming  for  the  general  officers  who 
were  making  these  decisions.  The  1988  process  was  simplified 
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due  to  efforts  by  the  Chief  of  the  Maintenance  Division  HQ 
TAC/SMO-R&M  (one  of  the  "Experts"  mentioned  earlier) .  The 
model  used  previously  was  reevaluated  using  only  objective 
data  from  P3B,  and  G063  data.  This  discussion  is  amplified 
in  the  validation  portion  of  the  next  chapter.  This 
improvement  allows  a  more  objective  assessment  of  the 
prioritization  process  and  provides  ease  of  model  use.  The 
prioritization  decision  listing  resulting  from  the  previous 
model  (1987)  was  duplicated  using  only  this  objective  data, 
thereby  validating  the  new  method. 

Using  this  as  a  baseline  for  the  1988  model,  the  revised 
mod  worksheet  shown  earlier  reflects  only  those  data  elements 
and  one  additional  data  element,  Aircraft  Impact.  (The 
rational  for  this  item  is  that  the  weapon  system  that  will 
experience  a  grounding  situation  should  the  mod  be  delayed, 
should  be  given  additional  priority  to  improve  the  ranking  of 
the  modification  in  the  process.) 

The  objective  data  from  the  mod  worksheets  is  input  into 
the  approved  model  by  HQ  TAC/JSG,  and  the  resulting  list  is 
provided  to  the  TAF  commanders  for  their  approval.  Once 
approved,  the  list  is  then  sent  to  HQ  AFLC  and  published  in 
Volume  II  of  the  R&M  2000  Plan. 

Though  this  is  an  improvement  to  last  year's  process, 
this  method  requires  additional  computer  analysts  support 
along  with  the  use  of  a  costly  mainframe  computer.  Inherent 
with  this  process  is  an  unnecessary  amount  of  time  to 
incorporate  a  change  to  the  model  or  correct  the  data. 


As  demonstrated  in  the  next  chapter,  flexibility  is  the 
key  to  the  improvement  to  the  process  designed  by  this 
author.  Through  software  available  for  the  microcomputers 
already  in  the  SMO,  the  process  will  be  readily  modified 
without  the  need  for  additional  support  from  the  Joint 
Studies  Group  at  Headquarters  TAC. 

Summary 

This  chapter  discussed  the  rationale  for  using  Action 
Research  as  the  preferred  method  to  research  this  process. 
This  method  of  research  allows  flexibility  in  the  research 
design  while  developing  solutions  for  "real  world" 
applications.  Also  discussed  in  this  chapter,  was  the 
process  HQ  TAC  uses  to  prioritize  the  Class  IV-B 
modifications.  While  the  improvements  made  by  the  Chief  of 
the  Maintenance  division  for  SMO-R&M  for  this  year's  model 
are  a  step  in  the  right  direction,  more  can  be  done.  The 
ability  of  the  process  to  be  placed  on  a  PC  will  greatly 
enhance  the  ability  of  the  managers  to  make  responsive 
changes  giving  the  decision  makers  remarkable  flexibility. 

In  addition,  the  decision  makers  will  benefit  from  the 
proposed  improvements  to  the  process,  specifically  the 
savings  of  time  and  resources  discussed  in  the  next  chapter. 
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IV.  Results  and  Discussion 


Overview 

This  chapter  discribes  the  author's  improvements  to  the 
TAF  Class  IV-B  prioritization  process,  development  and 
implementation  of  an  ESS,  and  validation  of  this  entire 
process.  Finally,  a  general  discussion  of  the  benefits 
derived  from  the  research  is  also  presented. 


The  Process 

An  expert  support  system  was  designed  by  the  author 
using  Expert  Choice,  developed  by  Decision  Support  Software. 
The  author  developed  a  three  step  approach  to  solve  the 
research  problem.  The  first  step  was  to  develop  a 
spreadsheet  to  analyze  the  input  data.  Entitled  the  data 
Analysis  Spreadsheet,  it  contained  the  raw  data  from  the  mod 
worksheets.  Figure  5  contains  sample  data  from  HQ'  TAC.  Data 
elements  (across  the  top  of  the  columns)  are  as  follows: 

1.  Aircraft  Impact  (A/C  Imp) 

2.  Reliability  Impact  (Rel  IMP) 

3.  Reliability  Rank  (Rel  R) 

4.  Maintainability  Impact  (M  I) 

5.  Maintainability  Rank  (M  R) 

6.  Aircraft  Year  Group  (A/C  YR  GP) 

7.  Inventory  Size  (Inv  Size) 

8.  Modification  Cost  (Mod  Cost) 

9.  Payback  Period 

Mod  names  are  shown  at  the  left  of  the  figure.  The  graph 
function  in  the  spreadsheet  software  allowed  the  data  to  be 
plotted  in  a  scatter-gram.  The  individual  data  elements  from 
each  mod  worksheet  were  plotted  in  the  graph  portion  of  the 
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Figure 

5. 

Data  Analysis  Spreadsheet 

Data  Elements 

Mod  Name 

SPM  1 

2 

3 

4 

5 

6 

7 

8 

9 

1.  F15EID 

1  4 

8.33 

16.56 

.26453 

9.24 

1979 

801 

26.03 

Yes 

2.  F16MBC 

1  3 

13.4 

57.55 

.20699 

8.21 

1981 

1278 

33.02 

No 

3.  A10SEC 

1  1 

29.4 

38 

.17234 

17.1 

1979 

274 

.357 

No 

4.  A7CFS 

1  2 

0 

10.34 

.0023 

8.15 

1979 

20 

4.65 

Yes 

5.  F111EMS 

2  3 

0 

9.29 

.18 

13.8 

1971 

277 

86.87 

No 

6.  GENG 

7.  . 

8.  . 

9.  . 

1  4 

etc . 

3 

24.12 

2.719 

100 

1983 

1200 

17.8 

Yes 

spreadsheet  software  to  determine  the  range  of  the  data  and 
whether  data  points  were  clustered  in  such  a  way  as  to  allow 
for  a  subdivision  into  data  ranges.  These  graphs  were  used 
to  identify  "natural"  break  points  and  develop  the  data 
ranges.  Data  ranges  were  then  extrapolated  from  the 
resulting  graphs  for  each  of  four  ranges  used  by  the  author 
and  a  table  (see  table  4)  was  created  for  rating  each  mod  on 
all  of  the  data  elements.  Once  defined,  these  data  ranges 
were  used  to  identify  a  numeric  value  from  one  (below 
average)  to  four  (outstanding)  for  each  of  the  data  elements 
on  the  worksheets . 

The  second  step  in  the  process  was  to  replicate  the  1988 
TAF  prioritization  process  described  in  the  last  chapter, 
using  the  microcomputer-based  ESS  Model  developed  by  the 
author  (see  figure  6) .  Notice  figure  6  is  very  similar  to 
figure  4  in  chapter  3.  The  SMO-R&M  experts  designed  the 
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Table 

4.  Data  Ranges 

Below 

Above  Out- 

Criteria 

Average 

Average 

Average  standing 

1. 

A/C  Impact 

(Based  on  the  experience 

of 

SMO-R&M) 

2. 

Rel  I 

0  -  0.49 

0.5  - 

1.99 

2 

- 

9.9  > 

10 

3. 

Rel  R 

0  -  9.9 

10  - 

20 

21 

- 

49  > 

50 

4. 

M  I 

0  -  0.05 

0.06  - 

1.5 

1.6 

- 

2.4  > 

2.5 

5. 

M  R 

0-4 

4.1  - 

9.9 

10 

- 

17  > 

18 

6. 

A/C  YR  GP 

<  1978 

1979  - 

1980 

1981 

- 

1982  > 

1983 

7. 

Inv  Size 

0  -  250 

251  - 

699 

700 

- 

1199  > 

1200 

8. 

Mod  Cost 

>  500 

499  - 

101 

100 

- 

16  < 

15 

9. 

Payback 

No 

Yes 

Period 

(Amortized 

or  Not) 

hierarchy  for  the  1988  model  and  assigned  the  respective 
weights  for  the  criteria  and  subcriteria.  This  entire 
process  was  duplicated  precisely  in  the  author's  ESS  model. 

The  only  difference  in  the  two  models  is  how  the  large 
number  of  alternatives  are  manipulated.  In  the  1988  model, 
the  mainframe  computer  manipulated  the  data.  The  Expert 
Choice  ESS  model,  run  on  a  microcomputer,  was  limited  in 
memory  capacity  available.  Four  alternatives  were  evaluated 
at  the  alternatives  level  for  each  of  the  data  elements  and 
each  mod  was  assigned  a  rating  from  "Below  average"  to 
"outstanding"  (arithmetically  1.0  to  4.0)  for  each  element, 
(see  Figure  6  and  refer  to  Appendix  A) 
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For  the  third  and  last  step,  the  author  developed 
another  spreadsheet  that  would  accumulate  the  points  awarded 
to  each  modification  for  each  of  the  data  element,  and  sort 
the  entire  list  of  modifications.  This  spreadsheet, 
designated  as  the  TAP  Class  IV-B  Mod  Spreadsheet,  was 
arranged  with  IF  . .  Then  . .  Else  statements  to  place  the 
values  for  each  of  the  data  elements  (see  figure  7) .  The 
numbers  derived  from  the  synthesis  of  the  entire  model  (see 
Appendix  B)  were  entered  into  the  spreadsheet  and  the  numeric 
values  (1.0  associated  with  below  average  etc.)  were  totaled 
into  a  specific  column  for  that  purpose.  For  example:  If  the 
Inventory  Size  data  element  of  a  particular  modification 
proposal  were  rated  1.0  from  the  data  ranges  (Table  4)  the 
spreadsheet  would  assign  a  value  of  0.006  to  the  total 
column;  a  rating  of  2.0  would  add  0.012;  a  3.0  would  add 
0.027;  and  a  rating  of  4.0  would  add  0.057  to  the  total 
column  (refer  to  Appendix  B  for  derivation  of  the  synthesis 


Figure 

7.  TAF 

Class 

IV- 

-B 

Mod 

Spreadsheet 

Mod  Name 

Total 

SPM 

1 

2 

3 

4 

5 

6 

7 

8 

9 

1. 

F15EID 

.312 

1 

4 

3 

2 

4 

2 

3 

3 

3 

3 

2. 

F16MBC 

.378 

1 

3 

4 

4 

3 

2 

3 

4 

3 

4 

3. 

A10GBAS 

.281 

1 

3 

1 

4 

2 

4 

1 

2 

4 

1 

4. 

A7CFS 

.083 

1 

2 

1 

2 

1 

2 

1 

1 

1 

1 

5. 

F111EMS 

.152 

2 

3 

1 

1 

3 

3 

1 

2 

2 

1 

6. 

Geng 

.414 

1 

4 

4 

3 

3 

4 

4 

4 

3 

2 

7. 

• 

8. 

• 

etc . 

9. 

• 

43 


values  used  here) .  The  sorted  list  was  then  compared  to  the 
1988  TAF  model  results.  The  mnemonic  name  associated  with 
each  of  the  proposed  modifications  allowed  consistent 
tracking  of  the  mods  between  the  two  spreadsheets,  the  mod 
worksheets,  and  the  ESS  model.  See  Appendix  A  and  B  for 
graphical  illustration  of  the  ESS  model  and  numeric  values 
provided  by  synthesis  of  judgments  to  each  data  element. 

Once  all  the  mods  were  ranked  and  entered  into  the  TAF 
Class  IV-B  Mod  Prioritization  spreadsheet,  it  was  a  simple 
matter  to  sort  the  spreadsheet  using  the  total  column  to 
produce  a  rank  ordered  mod  listing. 

Validation 

Although  there  are  known  limitations  to  action  research 
with  inferences  to  a  general  population,  validation  of  the 
model  was  possible.  The  most  notable  validation  of  the  model 
is  that  TAC  is  actually  planning  to  implement  the  author ' s 
ESS  immediately.  The  new  methodology  (the  ESS  model) ,  will 
be  used  to  provide  decision  makers  with  a  prioritized  list. 
This  in  and  of  itself  is  an  important  "validation"  of  the 
process . 

The  proposed  process  was  also  evaluated  by  the  Chief  of 
the  Special  Management  Office  for  R&M.  He  was  convinced  that 
the  process  would  save  a  considerable  amount  of  time, 
manpower,  mainframe  computer  time  —  and  therefore  money. 

In  addition,  the  improved  process  was  described  and 
demonstrated  to  the  Joint  Studies  Group  at  HQ  TAC.  They  were 
responsible  for  the  programming  of  the  original  model  that 
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was  used  in  the  past.  This  model  used  AHP  and  provided  the 
decision  makers  with  the  ability  to  make  the  pairwise 
comparisons  necessary  (in  the  previous  model)  to  rank  order 
the  mods,  but  was  mainframe-based  and  inflexible.  The  Chief 
analyst  responsible  for  the  JSG  effort  was  impressed  with  the 
ease  and  flexibility  of  the  modeling  environment  using  ESS. 
Following  the  demonstration,  the  JSG  analysts  discussed  the 
potential  for  the  PC-based  ESS  and  they  confirmed  the 
author's  belief  that  the  new  method  would  solve  the  problem 
without  the  use  of  a  mainframe  computer  support  (Schooff, 
1988). 

The  JSG  analyst  used  the  structure  of  the  1988  model  to 
validate  the  process  using  five  mods  from  the  1987  effort 
(see  figure  8).  The  resulting  prioritized  list  using  the 
mainframe  AHP  model  provided  the  same  results  as  was  derived 
through  the  process  used  by  the  decision  makers  in  the  1987 


Figure 

8. 

Mod  (1987) 

Priorities 

Mod  Name 

1 

2 

3 

4 

5 

6 

7 

8 

9 

16RDRI 

2 

2.1 

41 

1.37 

3 

1983 

1278 

4.27 

No 

16BATC 

1 

13.4 

1 

241 

28 

1983 

1278 

609 

No 

16FDR 

1 

0 

100 

0 

0 

1983 

1278 

227 

No 

16CSDG 

1 

15.8 

>100 

22.5 

33 

1983 

1278 

499 

No 

16RLG 

1 

15.2 

7 

4 

10 

1983 

1278 

91.4 

No 

Ratings 

16RDRI 

2 

1 

3 

1 

1 

4 

4 

1 

1 

16BATC 

1 

3 

4 

4 

2 

4 

4 

1 

1 

16  FDR 

1 

1 

1 

1 

1 

4 

4 

1 

1 

16CSDG 

1 

4 

1 

2 

3 

4 

4 

1 

3 

16RLG 

1 

1 

4 

3 

1 

4 

4 

3 

3 
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Figure 

9. 

Validation 

Nod  Name 

Total 

1 

2 

3 

4 

5 

6 

7 

8 

9 

16BATC 

.267 

1 

3 

4 

4 

2 

4 

4 

1 

1 

16RLG 

.185 

1 

1 

4 

3 

1 

4 

4 

3 

1 

16CSDG 

.173 

1 

4 

1 

2 

3 

4 

4 

1 

1 

16RDRI 

.143 

2 

1 

3 

1 

1 

4 

4 

4 

1 

16FDR 

.063 

1 

1 

1 

1 

1 

4 

4 

1 

1 

process.  These  same  five  mods  were  worked  into  the  author's 
ESS  model  and  the  resulting  list  was  the  same  as  that  derived 
in  the  previous  two  methods  (see  figure  9) . 

To  sum,  the  process  was  validated  three  ways:  the  first 
was  through  intended  use  in  an  operational  Air  Force  unit; 
the  second  by  the  JSG  computer  analysts  that  were  previously 
responsible  for  the  process;  and  last  by  a  data  trial  using 
data  from  last  year's  process. 

Discussion 

There  are  many  benefits  to  using  the  analytic  hierarchy 
process  approach  to  problem  solving,  and  for  implementing  the 
AHP  process  on  a  micro-computer.  The  first  is  inherent  with 
the  defining  and  ordering  of  the  problem.  The  problem,  once 
structured,  is  less  complex.  Merely  taking  the  time  to 
structure  the  problem  provides  insight  into  the  inexact  way 
the  typical  decision  is  made. 

The  second  benefit  has  to  do  with  focusing  the  attention 
to  a  portion  of  the  larger  problem.  This  allows  simpler, 
better  defined  responses  to  a  problem.  The  process  is 
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similar  to  the  problem  of  how  to  climb  a  mountain.  One 
starts  the  climb  with  the  first  steps  and  progresses  with 
each  additional  step.  To  observe  the  entire  mountain  the 
mind  cannot  fathom  how  to  progress  -  but  broken  down  and 
visualizing  each  step,  the  process  is  less  overwhelming. 

Finally,  the  ability  to  document  portions  of  the 
decision  process  allows  replication  of  the  process.  Should 
the  decision  be  questioned  at  any  time  the  ability  of  the 
decision  maker  to  document  and  replicate  the  process  is 
invaluable . 

Summary 

This  chapter  has  described  the  improvements  the  author 
has  made  to  TAC's  prioritization  process.  The  expert  support 
system  is  a  simple  process  for  solving  a  difficult  problem. 
Prior  to  expert  support  systems  like  this  one,  decisions  were 
often  based  largely  on  political  factors.  Traditional 
techniques  used  to  solve  these  nonprogrammed  decisions  are  to 
use  judgments,  intuition,  creativity,  and  rules  of  thumb. 
Clearly  with  multiple  criteria  and  more  subcriteria  these 
techniques  are  not  adequate  when  one  considers  that  millions 
of  tax  dollars  are  obligated  based  on  the  decision. 

The  ESS  developed  by  the  author  encourages  a  more 
rational  approach,  yet  still  capitalizes  on  traditional 
approaches.  The  difference  is  in  the  visibility  afforded 
into  all  factors  that  influence  the  decision.  And  the 
resulting  efficiency  introduced  into  the  decision  making 
process  has  great  cost-saving  potential. 


47 


V.  Recommendations 


Overview 

Headquarters  TAC/SMO-R&M  is  attempting  to  implement  a 
methodology  that  will  reduce  the  amount  of  time  needed  by  the 
top  echelons  of  Air  Force  managers  to  make  quality  decisions. 
These  decision  makers  need  a  decision  making  tool  to  help 
them  cope  with  complexity,  make  rational  decisions  within 
that  context,  and  defend  their  decisions  to  others  whose 
modifications  will  not  be  funded.  Currently  the  process  is 
accomplished  on  a  mainframe  computer  operated  by  HQ  TAC  Joint 
Studies  Group.  Their  computer  is  not  dedicated  solely  to  the 
prioritization  of  the  Class  IV-B  modifications.  With  the 
methods  and  techniques  discussed  in  this  research  the 
prioritization  process  could  be  accomplished  on  a 
microcomputer  located  in  the  HQ  TAC/SMO-R&M  office.  By  using 
current  in-place  equipment,  with  no  additional  manpower 
requirements,  the  prioritization  process  is  simplified,  more 
flexible,  and  more  responsive. 

Recommendations 

Further  research  is  needed  to  insure  validation  of  this 
process.  Using  only  one  organization  as  a  test  case 
resembles  the  one-shot  case  study  methodology  with  its  known 
threats  to  both  internal  and  external  validity.  While  this 
study,  using  action  research  methodology,  does  not  attempt  to 
overcome  validity  problems,  neither  does  it  claim  to  be 
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generalizable.  However,  it  is  the  author's  belief  that 
expert  support  systems  can  lead  to  improved  decision  making. 
More  organizations  and  further  modeling  processes  are  needed 
to  validate  this  claim.  It  is  the  opinion  of  those  involved 
with  this  research  that  there  are  many  opportunities  to 
appply  these  techniques  to  other  applications,  once  knowledge 
about  the  process  is  gained.  Many  organizations  are 
responsible  for  making  complex  decisions.  With  budget 
limitations  a  reality  many  complex  decisions  involve  limited 
resources.  Expert  support  systems  could  be  helpful  to  many 
organizations  striving  to  "do  more  with  less." 

Other  techniques  to  allow  multiple  alternatives  using 
Expert  Choice  came  to  the  attention  of  this  author  at  the  end 
of  this  research.  Later  versions  of  Expert  Choice  allow  the 
ESS  to  handle  a  large  number  of  alternatives  without  the  need 
to  use  the  external  spreadsheets.  This  would  have  greatly 
simplified  this  project.  More  research  is  needed  to 
determine  if  this  is  a  viable  method  of  prioritizing  these 
modifications.  Alas,  there  was  interest,  but  not  time  to 
develop  these  techniques . 

Summary 

This  research  was  intended  to  demonstrate  that  complex 
prioritization  decisions  can  be  handled  easily  by  a  PC  with 
inexpensive  software.  Air  Force  managers  and  leaders,  using 
the  demonstrated  software,  can  make  flexible,  reliable,  and 
responsive  decisions  even  in  the  nonprogrammed  decision 
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environment.  Dr.  Herbert  Simon  stated  in  his  1965  work.  The 
Shape  of  Automation. 

"With  operations  research  and  electronic  data 
processing  we  have  acquired  the  technical  capacity  to 
automate  programmed  decision  making  and  to  bring  into 
the  programmed  area  some  important  classes  of  decisions 
that  were  formerly  unprogrammed.  Important  innovations 
in  decision-making  processes  in  business  are  already 
resulting  from  these  discoveries. 

With  heuristic  programming,  we  are  acquiring  the 
technical  capacity  to  automate  nonprogrammed  decision 
making.  The  next  two  decades  will  see  changes  in 
business  decision  making  and  business  organization  that 
will  stem  from  this  second  phase  in  the  revolution  of 
our  information  technology.  (Simon,  1965:92)." 

Simon  was  not  far  off  in  his  time  assessment.  Similar  to 

private-sector  business  leaders,  AF  managers  and  leaders  face 

complex  decisions  in  a  constantly  changing  environment.  The 

use  of  computer  tools  like  the  expert  support  system 

described  in  this  research  can  use  available  Air  Force 

resources,  improve  responsiveness,  increase  flexibility  of 

the  process,  and  improve  the  reliability  of  the  decision 

making  process. 
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Select  the  mods  with  the  highest  R&M  improvement. 


LCC 

COST 

A/C  IMPT  - 
A/C  YRGP  - 
BENEFITS  - 
INV  SIZE  - 
MOD  COST  - 
R&M  BNFT  - 


G  0.101 


-  Life  cycle  cost  -  Amortized  or  not? 

-  Cost  of  the  total  projected  modification. 

-  Impact  on  the  mods  with  the  most  R&M  improvement 

-  Aircraft  Year  Group. 

-  Gained  by  the  Modification. 

-  Aircraft  Inventory  Size. 

-  Total  Dollar  Amount  Projected. 

-  Reliability  and  Maintainability  benefits. 


G 


GLOBAL  PRIORITY:  PRIORITY  RELATIVE  TO  GOAL 


A/C  IMPT 

R&M  BNFT 

A/C  YRGP 

G  0.154 

G  0.495 

G  0.050 

_ 1 

_ 1 

_ J 

—  OUTSTAND 
G  0.087 

—  ABV.  AVG 
G  0.040 

—  AVERAGE 

G  0.018 

L-  BELOW  AV 

—  RELIABTY 
G  0.247 

—  MAINTBTY 
G  0.247 

—  OUTSTAND 
G  0.028 

—  ABV.  AVG 
G  0.006 

—  AVERAGE 

G  0.006 

—  BELOW  AV 

G  -.009 


G  0.003 


OUT STAND 
G  0.057 
ABV.  AVG 
G  0.012 
AVERAGE 
G  0.012 
BELOW  AV 
G  0.003 


A/C  IMPT 
A/C  YRGP 
ABV.  AVG 
AVERAGE 
BELOW  AV 
BENEFITS 
INV  SIZE 
MAINTBTY 
OUTSTAND 
R&M  BNFT 
RELIABTY 


Impact  on  the  Aircraft  Fleet. 

Aircraft  Year  Group. 

Above  Average  Attributes. 

Average  Attributes. 

Below  Average  Attributes. 

Benefits  gained  by  the  Modification. 
Aircraft  Inventory  Size. 

Maintainability  factors. 

Outstanding  attributes. 

Reliability  and  Maintainability  Benefits. 
Reliability  Factors. 


G  -  GLOBAL  PRIORITY:  PRIORITY  RELATIVE  TO  GOAL 
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0 


G  0.124  G  0.124 


RANK  -  Rank  given  in  MODAS. 

IMPROV  H  -  Improvement  Hours. 

MAINTBTY  -  Maintainability  Factors. 

R&M  BNFT  -  Reliability  and  Maintainability  Benefits. 

RELIABTY  -  Reliability  Factors. 

G  -  GLOBAL  PRIORITY:  PRIORITY  RELATIVE  TO  GOAL 
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RANK 

ABV ,  AVG  - 
AVERAGE  - 
BELOW  AV  - 
IMPROV  H  - 
OUTSTAND  - 
RELIABTY  - 

G 


0  0  0  0 


RELIABTY 

G  0.247 

0 

IMPROV  H 

RANK 

G  0.124 

G  0.124 

OUTSTAND 
G  0.070 
ABV.  AVG 
G  0.032 
AVERAGE 
G  0.015 
BELOW  AV 
G  0.007 


OUTSTAND 
G  0.070 
ABV.  AVG 
G  0.032 
AVERAGE 
G  0.015 
BELOW  AV 
G  0.007 


-  Rank  Given  in  MODAS. 

-  Above  Average  Attributes. 

-  Average  Attributes. 

-  Below  Average  Attributes. 

-  Improvement  Hours. 

-  Outstanding  Attributes. 

-  Reliability  Factors. 

-  GLOBAL  PRIORITY:  PRIORITY  RELATIVE  TO  GOAL 
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RANK 

ABV.  AVG  - 
AVERAGE  - 
BELOW  AV  - 
IMPROV  H  - 
MAINTBTY  - 
OUT STAND  - 


0 


0  0  0  0 


G  0.007  G  0.007 


-  Rank  Given  in  MODAS. 

-  Above  Average  Attributes. 

-  Average  Attributes. 

-  Below  Average  Attributes. 

-  Improvement  Hours. 

-  Maintainability  Factors. 

-  Outstanding  Attributes. 


G 


—  GLOBAL  PRIORITY:  PRIORITY  RELATIVE  TO  GOAL 


o 


LCC 

COST 

ABV.  AVG  - 
AVERAGE  - 
BELOW  AV  - 
MOD  COST  - 
OUTSTAND  - 

G 


G  0.006 


G  0.006 


-  Life  Cycle  Cost  -  Amortized  or  not? 

-  Total  Cost  of  the  Projected  Modification. 

-  Above  Average  Attributes. 

-  Average  Attributes. 

-  Below  Average  Attributes. 

-  Total  Dollar  Amount. 

-  Outstanding  Attributes. 

-  GLOBAL  PRIORITY:  PRIORITY  RELATIVE  TO  GOAL 
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Select  the  roods  with’ the  highest  R&M  Improvement. 
TALLY  FOR  SYNTHESIS  OF  LEAF  NODES  WITH  RESPECT  TO  GOAL 


LEVEL  1 
BENEFITS 


LEVEL  2 

LEVEL  3 

LEVEL  4 

LEVEL  5 

0.800 

X 

R&M  BNFT 

=  0.495 

RELIABTY 

=  0.247 

IMPROV  H 

2 

0.124 

OUTSTAND 

= 

0.070 

ABV.  AVG 

= 

0.032 

AVERAGE 

= 

0.015 

BELOW  AV 

=r 

0.007 

RANK 

=t 

0.124 

OUTSTAND 

sz 

0.070 

ABV.  AVG 

s 

0.032 

AVERAGE 

s 

0.015 

BELOW  AV 

s 

0.007 

MAINTBTY 

»  0.247 

IMPROV  H 

* 

0.124 

OUTSTAND 

s 

0.070 

ABV.  AVG 

s 

0.032 

AVERAGE 

= 

0.015 

BELOW  AV 

s 

0.007 

RANK 

2 

0.124 

OUTSTAND 

a 

0.070 

ABV.  AVG 

ss 

0.032 

AVERAGE 

s 

0.015 

BELOW  AV 

s 

0.007 

A/C  IMPT  *  0.154 

OUTSTAND  =  0.087 
ABV.  AVG  =  0.040 
AVERAGE  -  0.018 
BELOW  AV  =  0.009 
INV  SIZE  «  0.101 

OUTSTAND  -  0.057 
ABV.  AVG  *  0.027 
AVERAGE  *  0.012 
BELOW  AV  «  0.006 
A/C  YRGP  ■  0.050 

OUTSTAND  -  0.028 
ABV.  AVG  -  0.013 
AVERAGE  *  0 . 006 
BELOW  AV  =  0.003 
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•  v 


COST  =  0.200 

LCC  =  0.100 

OUTSTAND  =  0.057 
ABV.  AVG  =  0.026 
AVERAGE  =  0.012 
BELOW  AV  =  0.006 
MOD  COST  «  0.100 

OUTSTAND  =  0.057 
ABV.  AVG  *  0.026 
AVERAGE  =  0.012 
BELOW  AV  =  0.006 


Select  the  mods  with  the  highest  R&M  improvement 

SYNTHESIS  OF  LEAF  NODES  WITH  RESPECT  TO  GOAL 

OVERALL  INCONSISTENCY  INDEX  *  0.01 

OUTSTAND 

ABV.  AVG 

AVERAGE 

BELOW  AV 

0.055  ...I  i  .. 

1.000 

ABV.  AVG 
AVERAGE 
BELOW  AV 
OUTSTAND 

-  Above  Average  Attributes. 

-  Average  Attributes. 

-  Below  Average  Attributes. 

-  Outstanding  Attributes. 

* 


k 
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GOAL:  Select  the  mods  with  the  highest  R&M  improvement. 


Glossary  of  the  Terms  Used  in  Model 

LCC  -  Life  Cycle  Costs  -  Amortized  or  not? 

COST  -  Cost  of  the  Total  Projected  Modifications. 

GOAL  -  Select  the  mods  with  the  highest  R&M  improvement 

RANK  -  Rank  Given  in  MODAS . 

A/C  IMPT  -  Impact  on  the  Aircraft  Fleet. 

A/C  YRGP  -  Aircraft  Year  Group. 

ABV .  AVG  -  Above  Average  Attributes. 

AVERAGE  -  Average  Attributes. 

BELOW  AV  -  Below  Average  Attributes. 

BENEFITS  -  Benefits  Gained  By  the  Modifications. 

IMPROV  H  -  Improvement  Hours. 

INV  SIZE  -  Inventory  Size. 

MAINTBTY  -  Maintainability  Factors. 

MOD  COST  -  Total  Dollar  Amount. 

OUTSTAND  -  Outstanding  Attributes. 

R&M  BNFT  -  Reliability  and  Maintainability  Benefits. 

RELIABTY  -  Reliability  Factors. 


* 
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A/C 

AFLC 

AHP 

AI 

ALC 

ANG 

Aircraft 

Air  Force  Logistics  Command 

Analytic  Hierarchy  Process 

Artificial  Intelligence 

Air  Logistic  Center 

Air  National  Guard 

DSS 

Decision  Support  System 

ES 

ESS 

Expert  System 

Expert  Support  System 

FH 

Flying  Hour 

G063 

Data  sheet  format  from  AFLC  computer 

system  MODAS 

HQ 

Headquarters 

ILS 

Imp 

Integrated  Logistic  Support 
Improvement 

JSG 

Joint  Studies  Group 

MIS 

MODAS 

Management  Information  System 

Maintenance  and  Operational  Data  Access  System 

PC 

POM 

P3B 

Personal  Computer 

Program  Objectives  Memorandum 

Data  from  AFLC  computer  system  Pacer 

Lab 

R&M 

Reliability  and  Maintainability 

SMO 

SPM 

Special  Management  Office 

Special  Project  Manager 

TAC 

TAF 

Tactical  Air  Command 
'Tactical  Air  Forces 

USAF 

USAFE 

United  States  Air  Force 

United  States  Air  Forces  in  Europe 
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